Data synthesis device and data synthesis method

ABSTRACT

A data combining apparatus and a data combining method in which timing for transformation, movement (arrangement), and generation of data combined in accordance with timing information can be corrected and a screen formation to be needed for matching of timing can be formed by combination, wherein a data combining apparatus  100  generates timing information of a change of the screen formation by a timing information generation unit  108  and synchronizes generation timing in the graphics generation unit  103  to process timing for image data in a transformation and movement processing unit  105  to enable combination of the image data and the graphics without timing deviation to thereby obtain desired combined data in a combining unit  107.

TECHNICAL FIELD

The present invention relates to a data combining apparatus preferablyused in such as a TV broadcast receiver, a TV monitor, a PC monitor, anediting apparatus, and a multimedia data reproduction apparatus forcombining still image data, moving image data, or graphics data to forma screen and a method for the same.

BACKGROUND ART

As a data combining apparatus for combining still image data, movingimage data, or graphics data to form a screen, there is known, forexample, a computer graphics system for synchronizing color data, depthdata, texture data, or other data processed asynchronously (refer toPatent Document 1: Japanese Unexamined Patent Publication (Kokai) No.9-6973) or a computer graphics system for combining a plurality ofthree-dimensional images in a two-dimensional graphics image processingpipeline and finally for performing rendering of the combined image on adisplay screen (refer to Patent Document 2: Japanese Unexamined PatentPublication (Kokai) No. 2001-357410).

FIG. 1 is a block diagram of an example of the configuration of aconventional data combining apparatus combining still image data, movingimage data, or graphics data etc. to form a screen.

The data formation apparatus 10, as shown in FIG. 1, has a screenformation control unit 11, a graphics control unit 12, a graphicsgeneration unit 13, an image control unit 14, a transformation andmovement control unit 15, an image processing unit 16, and a combiningunit 17.

The screen formation control unit 11 controls when and how data to becombined should be transformed (enlarged or reduced) and where theresult should be moved (arranged) and combined on a screen.

The graphics control unit 12 controls the graphics generation unit 13and generates graphics to be needed for a combined screen instructedfrom the screen formation control unit 11. The generated graphics S13 tobe combined is input to the combining unit 17.

On the other hand, the image control unit 14 controls the transformationand movement processing unit 15 of the image and transforms and moves aninput image data SIM to correspond to the combined screen instructedfrom the screen formation control unit 11. The transformed and movedimage data S15 may be further processed by the image processing unit 16.

The image processing unit 16 performs, for example, filtering for animprovement of image quality and conversion of the image format. Notethat, the image processing unit 16 sometimes is not provided. Also,similarly, a graphics processing unit not shown in the figures may beplaced at the latter stage of the graphics generation unit 13.

The transformed and moved image data S15 is input to the combining unit17 as image data S16 to be combined.

The combining unit 17 combines the input graphics S13 to be combined andthe input image data S16 to be combined to output the combined data S17.

For example, a TV monitor or a PC monitor forms a menu provided to theuser as a graphics, superposes the same on image data to be displayed,and displays the result. Such a system has also been called an“on-screen-display (OSD)”.

Further, the data combining apparatus shown in FIG. 1 is used in aservice such as digital television broadcasting in which image data andgraphics data are transmitted and a screen formation formed by combiningthe both is provided.

On the other hand, the combination system 10 shown in FIG. 1 does notstrictly adjust the timing of the graphics S13 to be combined and theimage data S16 to be combined.

Therefore, in the conventional data combining apparatus, there aredisadvantages that it cannot accurately realize a screen formation suchthat transformation and movement (arrangement) of the data to becombined are included and specifically such that the amount oftransformation and movement and arrangement position changecontinuously.

This point will be further considered with reference to FIG. 2A to FIG.2C and FIG. 3A to FIG. 3C.

FIG. 2A to FIG. 2C are views showing an example of combination of thescreen formation such that transformation and movement (arrangement) ofthe data to be combined are included and specifically the amount oftransformation and movement and the arrangement position changecontinuously. FIG. 2A shows the graphics S13 to be combined, FIG. 2Bshows the image data S16 to be combined, and FIG. 2C shows the combineddata S17 respectively.

As shown in the figures, at the time T1, the transformation (reduction)and movement of the image data S16 to be combined is started.

In accordance with image data to be moved after the reduction, availablegraphics is generated at a region in which the image data does notexist.

In the example shown in FIG. 2A to FIG. 2C, the image data S16 and S17are continuously reduced and moved along with the elapse. The availablegraphics region changes due to this.

As long as the image data S16 to be combined and the graphics S13 to becombined are combined at an accurate timing, the two regions are madecorrespondence accurately.

On the other hand, if the timing of the transformation and movement ofthe image data and the timing of the generation of the graphics deviatefrom each other, desirable combined data cannot be achieved.

FIG. 3A to FIG. 3C are views showing an example of combination in thecase where the start of the transformation and movement of the imagedata is delayed by one frame's worth of time. FIG. 3A shows the graphicsS13 to be combined, FIG. 3B shows the image data S16 to be combined, andFIG. 3C shows the combined data S17 respectively.

As shown in the figures, at the time T1, the change in the screenformation is started, and the graphics is generated at the accuratetiming.

The example shown in FIG. 3A to FIG. 3C shows an example in which thestart of the transformation (reduction) and movement of the image datato be combined is delayed at the time T2.

At the time T1, the image data S16 in which the change of the screenformation has not been started and the graphics S13 in which the samehas been started are input to the combining unit 17, so thatcontradictory combined data is output.

Generally, since the graphics is superposed and combined at the front ofthe image data in the OSD, FIG. 3A to FIG. 3C show the case ofsuperposing the graphics at the front of the image data. Similarly tothis, in the case of superposing the image data at the front of thegraphics, the contradictory combined data is output.

Further, the delay of the start of the transformation (reduction) andmovement of the image data to be combined is propagated to subsequenttimings of combination and consequently the combination is successivelyperformed with the deviated timing of the screen formation.

This is because, as described above, in the conventional data combiningapparatus combining image data and graphics data etc. to form thescreen, the accurately timing adjustment for combining the graphics tobe combined and the image data to be combined does not performed.

Particularly, in the case where each of the control units (the screenformation control unit, the image control unit, and the graphics controlunit) is configured by a plurality of processors, in the case wherecommunication paths between the control units or between a control unitand a processing unit (the transformation and movement processing unitor the graphics generation unit) are connected by buses including delay,or in the case where the communication takes time, the delay easilyoccurs until a notice of change of the screen formation is actuallyreflected to the transformation and movement processing unit or thegraphics generation unit.

Also, in the case where each of the control units or processing units isconfigured by software, the delay easily occurs.

Further, if the image data and the graphics are asynchronous, the casewhere the timings of the two are deviated is similar to the case whereany one of them is delayed.

Therefore, in the conventional data combining apparatus, it is difficultto realize a screen formation including transformation and movement(arrangement) of the data to be combined and specifically needing toadjust the timing such that the transformation and amount of movementand the arrangement position are changed continuously.

Namely, in the OSD, restrictions that the graphics and the image data tobe combined have no relation in time are imposed.

In a screen formation combining a graphics at a region without thetransformed and moved (arranged) image data, use is made under therestriction that the screen formation be one in which the amount oftransformation and movement and the arrangement position do not changecontinuously.

Under the latter restriction, when the screen formation is switched, dueto the deviation of timing of the image data and the graphics, acontradictory combined image is displayed.

DISCLOSURE OF THE INVENTION

An object of the present invention is to provide a data combiningapparatus capable of correcting timing for transformation, movement(arrangement), or generation of data combined in accordance with thetiming information and capable of combining a screen formation to beneeded for adjustment of a timing and a data combining method of thesame.

To achieve the object, according to a first aspect of the presentinvention, there is provided a data combining apparatus combining firstdata and second data relating to an image to form a screen, theapparatus having: a processing unit for performing predeterminedprocessing on the first data; a data generation unit for generating thesecond data; a combining unit for combining the first data processed bythe predetermined processing in the processing unit and the second datagenerated by the data generation unit; a timing information generationunit for generating timing information for the processing of theprocessing unit or for processing of a generation of the data generationunit so that the combination of the first data processed by thepredetermined processing in the processing unit and the second datagenerated by the generation unit is performed at the same timing; and acontrol unit for correcting the processing timing or generation timingto make the processing unit or the data generation unit perform thepredetermined processing or the data generation based on the timinginformation generated by the timing information generation unit.

A second aspect of the present invention provides a data combiningapparatus combining first data and second data relating to an image toform a screen, the apparatus having: a processing unit for performingpredetermined processing on the first data; a data generation unit forgenerating the second data; a combining unit for combining the firstdata processed by the predetermined processing in the processing unitand the second data generated by the data generation unit; and a timinginformation generation unit for generating at least one of the timinginformation of the processing in the processing unit and the processingfor a generation in the data processing unit so that the combination ofthe first data processed by the predetermined processing in theprocessing unit and the second data generated by the generation unit isperformed at the same timing in the combining unit. The combining unitcorrects the two data to perform the combination at the same timingbased on the timing information generated by the timing informationgeneration unit.

Preferably, the data combining apparatus has a second processing unitperforming processing to introduce delay to the first data processed bythe predetermined processing by the processing unit, and the timinginformation generation unit adds an amount of delay in the secondprocessing unit and generates the timing information indicating an inputtiming to the combining unit.

Preferably, the timing information generation unit generates the timinginformation including a countdown up to a start of a change of screenformation.

Further, preferably, a data combining apparatus further has asuperposition unit for superposing the timing information generated bythe timing information generation unit to the first data or the seconddata, and corrects the two data to perform the combination at the sametiming based on the superposed timing information.

Preferably, the superposition unit superposes the timing information ata blanking period of the first data or the second data relating to theimage.

A third aspect of the present invention provides a data combining methodfor combining first data and second data relating to an image to form ascreen, the method having: a first step of performing predeterminedprocessing on the first data; a second step of generating the seconddata; and a third step of combining the first data processed by thepredetermined processing at the first step and the second data generatedby the data generation unit, and the timing information for theprocessing at the first step or the processing for generation at thesecond step is generated so that the combination of the first dataprocessed by the predetermined processing at the first step and thesecond data generated at the second step is performed at the sametiming, and at the first step or at the second step, the timing ofprocessing or generation is corrected based on the timing information toperform the predetermined processing or data generation.

A fourth aspect of the present invention provides a data combiningmethod combining first data and second data relating to an image to forma screen, the method having: a first step of performing predeterminingprocessing on the first data; a second step of generating the seconddata; and a third step of combining the first data processed by thepredetermined processing at the first step and the second data generatedby the data generation unit, at least one of the timing information forthe processing at the first step and the generation processing at thesecond step being generated so that the combination of the first dataprocessed by the predetermined processing at the first step and thesecond data generated at the second step is performed at the same timingin the third step, and at the third step, the combination beingcorrected based on the timing information so as to be performed at thesame timing.

According to the present invention, for example, the transformation andmovement processing unit for the image data as the first data or thesecond data or the generation unit for graphics data as the second dataor the first data generates the timing information for the screenformation and corrects the timing of the transformation and movementprocessing or the generation processing for one set of data based on thetiming information of another set of data to match with the timing ofthe two data.

Further, according to the present invention, the transformation andmovement processing unit for the image data as the first data or thesecond data and/or the generation unit for the graphics as the second orthe first data generates the timing information for the screen formationand combines the data at the corresponding timing based on the twotiming information to make the timings of the two data match.

According to the present invention, since the correction may beperformed based on the timing information so that the two data of thecorresponding timings are combined and output even if the timing ofstart of the screen formation is delayed or if it is early or adeviation in timing occurs in the middle of a change of the screenformation, desired combined data is obtained in the screen formationincluding transformation and movement (arrangement) of data to becombined and specifically needing to adjust the timing so that theamount of transformation and movement and the arrangement positionchange continuously.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of the configuration of a conventional datacombining apparatus.

FIG. 2A to FIG. 2C are views showing an example of timing of combinationof a graphics and image data.

FIG. 3A to FIG. 3C are views for explaining a disadvantage in the casewhere the timing of combination is deviated in the conventional datacombining apparatus.

FIG. 4 is a block diagram of a first embodiment of a data combiningapparatus according to the present invention.

FIG. 5A to FIG. 5D are views for explaining the case in the firstembodiment where an image processing unit has no delay and a graphicscontrol unit and a graphics generation unit can respond to the timinginformation immediately.

FIG. 6A to FIG. 6D are views for explaining the case in the firstembodiment where delay is needed until the graphics control unit and thegraphics generation unit reflect the timing information.

FIG. 7A to FIG. 7D are views for explaining the case in the firstembodiment where a countdown is introduced to the timing information tomake the timing of the graphics and that of the image data match all thetime even when the graphics control unit and the graphics generationunit need delay until the timing information is reflected.

FIG. 8 is a block diagram of a second embodiment of the data combiningapparatus according to the present invention.

FIG. 9 is a block diagram of a third embodiment of the data combiningapparatus according to the present invention.

FIG. 10A to FIG. 10E are views for explaining the timing information inthe third embodiment and a correction operation in a combining unit.

FIG. 11 is a view for explaining the processing from generation of thetiming information to detection thereof in the case of superposing thetiming information on data.

BEST MODE FOR CARRYING OUT THE INVENTION

Below, embodiments of the present invention will be explained withreference to the attached drawings

First Embodiment

FIG. 4 is a block diagram showing a first embodiment of a data combiningapparatus according to the present invention.

The data combining apparatus 100 generates timing information for achange of a screen formation and matches the timing of generation forgraphics with the timing of transformation and movement processing onimage data as first data or second data so as to enable combination ofgraphics as the second data or the first data without timing deviation(timing difference) and the image data to thereby obtain desiredcombined data.

The data combining apparatus 100, as shown in FIG. 4, has a screenformation control unit 101, a graphics control unit 102, a graphicsgeneration unit 103, an image control unit 104, a transformation andmovement processing unit 105 as a processing unit, an image processingunit 106 as a second processing unit, a combining unit 107, and a timinginformation generation unit 108 of image data.

The screen formation control unit 101 controls when and how data to becombined should be transformed (enlarged or reduced) and where theresult should be moved (arranged) and combined on a screen.

The graphics control unit 102 detects and corrects the timing deviationof the image data based on the timing information S108 of the change ofthe screen formation by the timing information generation unit 108 andcontrols the graphics generation unit 103 to generate the graphics to beneeded for the combined screen instructed from the screen formationcontrol unit 101 so as to match the corrected timing based on the timinginformation S108.

The graphics generation unit 103 generates the graphics to be needed forthe combined screen instructed from the screen formation control unit101 under the control of the graphics control unit 102 in accordancewith the timing information S108, then outputs the result as graphicsS103 to be combined to the combining unit 107.

Note that, a graphics processing unit not shown in the figures may bearranged at the latter stage of the graphics generation unit 13.

The image control unit 104 controls the transformation and movementprocessing unit 105 for the image and transforms and moves input imagedata SIM so as to correspond to the combined screen instructed from thescreen formation control unit 101.

The transformation and movement processing unit 105 transforms and movesthe input image data SIM under the control of the image control unit 104so as to correspond to the combined screen in accordance with theinstruction from the screen formation control unit 101 and outputstransformed and moved image data S105, for example, to the imageprocessing unit 106.

The image processing unit 106 performs, for example, filtering forimprovement of image quality and conversion of the image format on thetransformed and moved image data S105 from the transformation andmovement processing unit 105 and outputs the result as image data S106to be combined to the combining unit 107.

Note that the image processing unit 106 sometimes is not provided.Similarly, a graphics processing unit not shown in the figures may bearranged at the latter stage of the graphics generation unit 13.

The combining unit 107 combines the input graphics S103 to be combinedand the image data S16 to be combined and outputs combined data S107.

The timing information generation unit 108 is placed in thetransformation and movement processing unit 105 of the image data,generates the timing information S108 for the change of the screenformation, and outputs the same to the graphics control unit 102.

The timing information S108 generated by the timing informationgeneration unit 108 generates the timing information in accordance with,for example, a first case where the image processing unit has no delayand the graphics control unit and the graphics generation unit canimmediately respond to the timing information, a second case where thegraphics control unit 102 and the graphics generation unit 103 needdelay until the timing information S108 is reflected, or a third casewhere the graphics control unit 102 and the graphics generation unit 103need delay until the timing information S108 is reflected due tointroduction of a countdown to the timing information.

Hereinafter, control operations based on the timing information at thefirst case, the second case, and the third case will be successivelyexplained with reference to the figures.

First, with reference to FIG. 5A to FIG. 5D, the control operation basedon the timing information in the first case will be explained.

FIG. 5A to FIG. 5D are views for explaining the case where the imageprocessing unit has no delay and the graphics control unit and thegraphics generation unit can immediately respond to the timinginformation.

FIG. 5A shows the graphics S103 to be combined, FIG. 5B shows the imagedata S106 to be combined, FIG. 5C shows the combined data S107, and theFIG. 5D shows the timing information S108, respectively.

In this case, the screen formation control unit 101 controls the changeof the screen formation to be performed from the time T1.

However, the start of the transformation and movement processing of theimage data by the transformation and movement processing unit 105 isdelayed and the processing is started at the time T2.

Therefore, the timing information S108 at the time T1 indicates“not-start (in an example shown in FIG. 5D, “0”)”.

The graphics control unit 102 normally can start the change of thescreen formation at the time T1 as shown in FIG. 2A to FIG. 2C. However,in the present embodiment, it does not start the same at the time Tbased on the timing information.

Therefore, the graphics S103 to be combined and the image data S106 tobe combined at the time T1 are combined without any contradiction in thescreen formation such as in the conventional case shown in FIG. 3A toFIG. 3C.

At the time T2, the transformation and movement processing of the imagedata is started by the transformation and movement processing unit 105,so the timing information S108 at the time T2 indicates “start” (in theexample shown in FIG. 5D, “1”).

The graphics control unit 102 starts the change of the screen formationat the time T2 based on the timing information S108. After the start ofthe change of the screen formation, use of the timing information S108enables combination while avoiding contradictions in the screenformation such as the conventional case shown in FIG. 3A to FIG. 3C.

Next, with reference to FIG. 6A to FIG. 6D, the control operation basedon the timing information in the second case will be explained.

FIG. 6A to FIG. 6D are views for explaining the case where the graphicscontrol unit 102 and the graphics generation unit 103 need delay untilthe timing information S108 is reflected.

FIG. 6A shows the graphics S103 to be combined, FIG. 6B shows the imagedata S106 to be combined, FIG. 6C shows the combined data S107, and theFIG. 6D shows the timing information S108, respectively.

In this case, similar to the first case described above, the screenformation control unit 101 controls the change of the screen formationto perform the same from the time T1, however, the start of thetransformation and movement processing of the image data performed bythe transformation and movement processing unit 105 is delayed, so theprocessing starts at the time T2.

The graphics control unit 102 has a delay until the timing informationS108 is reflected and starts the change of the screen formation at timeT1 based on the instruction from the screen formation control unit 101.

Therefore, the combined result with respect to an object to be combinedat the time T1 becomes combined data having a contradiction in thescreen formation similar to the conventional case in FIG. 3A to FIG. 3C.

However, at the time T2, the graphics control unit 102 becomes possibleto reflect the timing information 115 at the time T1. The timinginformation S108 at the time T1 indicates “not start (in the exampleshown in FIG. 6D, “0”)”, so a graphics stopping the change of the screenformation is generated.

At the time T3, the graphics control unit 102 refers to the timinginformation S108 at the time T2 and then learns that the transformationand movement processing of the image data by the transformation andmovement processing unit 105 has started (in the example of the timinginformation shown in FIG. 6D, “1”) at the time T2, so it restarts thechange of the screen formation for the graphics to avoid contradictionin the screen formation.

After the time T2, by using the timing information S108, it is possibleto combine them while avoiding contradiction in the screen formationsuch as the prior art shown in FIG. 3A to FIG. 3C.

Note that, if a side where delay can occur (the image data in theexample shown in FIG. 6A to FIG. 6D) is displayed at the entire screenregion before the change of the screen formation, by combining thescreen formation by making the side where delay can occur the front, thecombination can be performed so as not to affect the contradiction inthe screen formation able to occur at the start of the change of thescreen formation to the combined data.

Next, with reference to FIG. 7A to FIG. 7D, the control operation basedon the timing information in the third case will be explained.

FIG. 7A to FIG. 7D are views for explaining the case where a countdownis introduced to the timing information so as to make the timing of thegraphics and that of the image data match from the start of the changeof the screen formation at all times even if the graphics control unit102 and the graphics generation unit 102 need delay until the timinginformation 115 is reflected.

FIG. 7A shows the graphics S103 to be combined, FIG. 7B shows the imagedata S106 to the combined, FIG. 7C shows the combined data S107, and theFIG. 7D shows the timing information S108, respectively.

In this case, in the same way as the first case, the screen formationcontrol unit 101 controls the change of the screen formation to performthe same at time T1 and additionally notifies the image control unit 104at the time 0 in advance.

In the example shown in FIG. 7A to FIG. 7D, the start of thetransformation and movement processing of the image data performed bythe transformation and movement processing unit 105 is delayed, so thetiming information S108 at the time TO remains “0 (not started and notprepared)”.

At the time T1, the graphics control unit 102 refers to the timinginformation at the time T0 and then leans “not started” by the timinginformation, so the graphics control unit 102 does not start the changeof the screen formation.

The timing information S108 of the image data indicates “−1 (startcountdown indicating to start at the following time)” at the time T1.The start countdown indicates a countdown up to a start time for thechange of the screen formation of the image data.

At the time T2, the graphics control unit 102 refers to the timinginformation S108 in time T1 and then learns the start at the time T2 inadvance due to “−1 (start countdown indicating to start at the followingtime)” of the timing information.

Therefore, the change of the screen formation is started at the time T2for both the image data and the graphics. By introducing the countdownto the timing information S108 as described above, it becomes possibleto make the timing of the graphics and that of the image data to bematched at all times even if the start of transformation and movementprocessing of the image data is delayed.

The explanations were made that cases without delay in the imageprocessing unit 106 with reference to FIG. 5A to FIG. 5D, FIG. 6A toFIG. 6D, and FIG. 7A to FIG. 7D. When the data combining apparatus 100is provided with an image processing unit 106 including delay, by makingthe timing information generation unit 108 consider the delay in theimage processing unit 106 and output the timing information S108corresponding to the time input to the combining unit 107, the presentinvention can be applied in the same way as when there is no delay inthe image processing unit 106.

The explanations were made that cases where the start of thetransformation and movement processing for the image data is delayed atthe point of the start of the change of the screen formation withreference to FIG. 5A to FIG. 5D, FIG. 6A to FIG. 6D, and FIG. 7A to FIG.7D. Further, when the start of the transformation and movementprocessing for the image data is advanced, it is clear that thecorrection can be performed by using the timing information S108 in thesame way as when the start is delayed.

The explanations were made that cases where the change of the screenformation of the image data and the graphics is deviated at the point ofstart of the change with reference to FIG. 5A to FIG. 5D, FIG. 6A toFIG. 6D, and FIG. 7A to FIG. 7D. Further, when the deviation occurs inthe middle of the change or when the deviation occurs at the end of thechange, it is clear that the correction can be performed in the same wayas when the deviation occurs at the point of the start of the change.

Second Embodiment

FIG. 8 is a block diagram of a second embodiment of a data combiningapparatus according to the present invention.

The point at which the second embodiment differs from the firstembodiment will be described below.

Namely, in the first embodiment, the transformation and movementprocessing unit 105 of the image data is provided with the timinginformation generation unit 108, and the graphics control unit 102detects and corrects a timing deviation based on the timing informationS108.

On the other hand, in the second embodiment, contrary to the above, agraphics generation unit 103A is provided with a timing informationgeneration unit 109, and an image control unit 104A detects and correctsthe timing deviation based on the timing information S109 and controls atransformation and movement processing unit 105A so that thetransformation and movement processing of the image data is performedunder the corrected timing.

The rest of the components are the same as those of the first embodimentdescribed above.

If replacing the timing of the graphics to be combined in FIG. 5A toFIG. 5D, FIG. 6A to FIG. 6D, and FIG. 7A to FIG. 7D with the timing ofthe image data to be combined, by making the image control unit 104Adetect and correct the timing deviation based on the timing informationin the same way as in the first embodiment, it is obvious that theeffect of the deviation in timing for generating the graphics can becorrected.

Third Embodiment

FIG. 9 is a block diagram of a third embodiment of a data combiningapparatus according to the present invention.

The point at which the third embodiment differs from the first andsecond embodiments will be described below.

Namely, in the first and second embodiment, the timing information ofeither the image data or the graphics is generated. The control unit forthe object to be combined refers to the timing information and changesthe timing of the transformation and movement processing or that ofgeneration of the graphics to correct.

On the other hand, the data combining apparatus 100B in the thirdembodiment generates the timing information S108 and S109 for both theimage data and the graphics and inputs the result to the combining unit107B.

Then, the combining unit 107B refers to the timing information S108 andS109 of the data to be combined and delays and combines the inputs to becombined using the frame memory 110 so that the timings of the data tobe combined match.

Next, the timing information in the third embodiment and the correctionoperation in the combining unit will be explained with reference to FIG.10A to FIG. 10E.

FIG. 10A shows the graphics S103 to be combined, FIG. 10B shows theimage data S106 to be combined, FIG. 10C shows the combined data S107,FIG. 10D shows the timing information S109 of the graphics, and FIG. 10Eshows the timing information S108 of the image data, respectively.

The screen formation control unit 101 controls the change of the screenformation to perform the same at the time T1. The graphics is startedwith the change of the screen formation at the time T1, so the timinginformation S109 of the graphics at the time T1 indicates “start (in theexample shown in FIG. 10D, “1”).

On the other hand, the start of the transformation and movementprocessing for the image data by the transformation and movementprocessing unit 105 is delayed, so the processing starts at time T2.

Therefore, the timing information S108 of the image data at time T1indicates “not start (in the example shown in FIG. 10E, “0”).

The combining unit 107B compares the two of the timing information andcan detect that the change of the screen formation of the image dataS106 to be combined is not started at the time T1.

In the example shown in FIG. 10A to FIG. 10E, at the time T1, only theimage data S106 to be combined is output as the combined data, but itmay be stored with the graphics S103 to be combined at the time T0 inthe frame memory 110 and combined with the image data S106 to becombined at the time T1.

Alternatively, it may be stored with the combined data S107 at the timeT0 in the frame memory 110 and output with the same repeatedly at thetime T1.

The combining unit 107B detecting deviation of the timing information atthe time T1 stores the data to be combined which is advanced in timing(in the example shown in FIG. 10A to FIG. 10E, the graphics S103 to becombined) on the frame memory 110 to delay the same.

The timing information S109 of the graphics at the time T2 indicates thesecond frame after start, while the timing information S108 for theimage data indicates the first frame after start. Based on the timinginformation, the combining unit 107B can detect the delay of one frame'sworth of the image data S106.

Therefore, in the combination at the time T2, the graphics S103 to becombined at the time T1 stored on the frame memory 110 is used to delaythe graphics and correct it so as to match with the timing of the imagedata S106.

After that, a similar operation may be used to combine graphics andimage data matched in timing at all times.

Further, in the third embodiment as well, by adding a countdown up tothe start of the change of the screen formation to the timinginformation, the image timing information at the time T1 shown in FIG.10A to FIG. 10E indicates preparation for start, so the combining unitcan learn that the graphics may be delayed by one frame's worth of timeat the time T1 in advance.

Next, the method of transfer of different timing information will beexplained with reference to FIG. 11.

In FIG. 11, 111 indicates a timing information generation unit, 112indicates a timing information detection unit, and 113 indicates anaddition unit as the superposing part, respectively.

The data combining apparatus 100, 100A, and 100B shown in FIG. 4, FIG.8, and FIG. 9 transfer the timing information separately from thegraphics and the image data.

The method of transfer of the timing information shown in FIG. 11 is tosuperpose the timing information S111 to the image data or the graphicsS120 in the addition unit 113 and detect the superposed timinginformation in the image data or the graphics included in the superposeddata S113 by the timing information detection unit 112.

According to this method of transfer, no communicating means fortransferring the timing information separately from the data isnecessary.

In particular, this method has an advantage that when the data isdelayed by the image processing unit 106, the timing information can bedelayed without any additional processing performed by the imageprocessing unit 106.

In the first and second embodiments described above as well, it ispossible to apply the method of transfer superposing the timinginformation to the image data or the graphics by making the timinginformation detection unit 112 notify the corresponding control unit ofthe timing information.

In particular, it is possible to transfer timing information without anyeffect on the graphics and the image data by superposing the timinginformation on a blanking period of the graphics and the image data(portion outside of available region for the screen formation).

As described above, according to the present invention, there is theadvantage that the combination of a screen formation to be needed formatching of the timing becomes possible by generating the timinginformation of, for example transformation, movement (arrangement), andgeneration of data forming the screen and correcting the timing fortransformation, movement (arrangement), and generation of data to becombined in accordance with the timing information.

Further, according to the present invention, the combination of a screenformation to be needed for matching of the timing becomes possible bygenerating the timing information of, for example, transformation,movement (arrangement), and generation of data forming the screen,comparing the timing information of the data to be combined in acombination stage, and correcting the timing of combination.

Further, by adding a countdown up to the start of the change of thescreen formation to the timing information, it is possible to correctthe timing without contradiction from the point of start of the changeof the screen formation even in a system where the timing informationcannot be immediately reflected.

Further, by superposing the timing information in the data to becombined, a communicating part for transferring the timing informationseparately from the data can be made unnecessary.

In particular, if the control units are configured by a plurality ofprocessors, if communication path between the control units or between acontrol unit and a processing unit is connected by a bus includingdelay, if the communication takes time, if the control units and theprocessing unit are configured by software, or if a plurality of data tobe combined are input asynchronously, according to the presentinvention, a desired screen formation can be formed by combination freefrom contradiction.

INDUSTRIAL APPLICABILITY

According to the data combining apparatus and data combining method ofthe present invention, the timing of transformation, movement(arrangement), and generation of data combined in accordance with thetiming information can be corrected and the screen formation to beneeded for matching of the timing can be formed by combination,consequently the invention can be applied to a TV broadcast receiver, aTV monitor, a PC monitor, an editing apparatus, and a multimedia datareproduction apparatus which combine still image data, moving imagedata, and graphics etc. to form a screen.

1. A data combining apparatus combining first data and second datarelating to an image to form a screen comprising: a processing unit forperforming predetermining processing on the first data; a datageneration unit for generating the second data; a combining unit forcombining the first data processed by the predetermined processing inthe processing unit and the second data generated by the data generationunit; a timing information generation unit for generating timinginformation for the processing of the processing unit or for processingfor generation of the data generation unit so that the combination ofthe first data processed by the predetermined processing in theprocessing unit and the second data generated by the generation unit isperformed at the same timing; and a control unit for correcting a timingfor processing or generation to make the processing unit or the datageneration unit perform the predetermined processing or data generationbased on the timing information generated by the timing informationgeneration unit.
 2. A data combining apparatus as set forth in claim 1,further comprising a second processing unit for performing processingintroducing a delay to the first data processed by the predeterminedprocessing by the processing unit, and wherein the timing informationgeneration unit adds an amount of delay in the second processing unitand generates timing information indicating an input timing to thecombining unit.
 3. A data combining apparatus as set forth in claim 1,wherein the timing information generation unit generates timinginformation including a countdown up to start of a change of screenformation.
 4. A data combining apparatus as set forth in claim 1,further comprising a superposing unit for superposing the timinginformation generated by the timing information generation unit to thefirst data or the second data, and wherein the control unit makes theprocessing unit or the data generation unit correct the timing ofprocessing or generation and perform the predetermined processing or thedata generation based on the superposed timing information.
 5. A datacombining apparatus as set forth in claim 4, wherein the superposingunit superposes the timing information on a blanking period of the firstdata or the second data relating to an image.
 6. A data combiningapparatus combining first data and second data relating to an image toform a screen comprising: a processing unit for performingpredetermining processing on the first data; a data generation unit forgenerating the second data; a combining unit for combining the firstdata processed by the predetermined processing in the processing unitand the second data generated by the data generation unit; and a timinginformation generation unit for generating at least one of the timinginformation of the processing in the processing unit and the processingfor generation in the data processing unit so that the combination ofthe first data processed by the predetermined processing in theprocessing unit and the second data generated by the generation unit isperformed at the same timing in the combining unit, wherein thecombining unit corrects and combines the data at the same timing basedon the timing information generated by the timing information generationunit.
 7. A data combining apparatus as set forth in claim 6, furthercomprising a second processing unit for performing processingintroducing a delay to the first data processed by the predeterminedprocessing by the processing unit, and wherein the timing informationgeneration unit adds an amount of delay in the second processing unitand generates the timing information indicating an input timing to thecombining unit.
 8. A data combining apparatus as set forth in claim 6,wherein the timing information generation unit generates timinginformation including a countdown up to a start of a change of screenformation.
 9. A data combining apparatus as set forth in claim 6,further comprising a superposing unit for superposing timing informationgenerated by the timing information generation unit to the first data orthe second data, and wherein the combining unit corrects and combinesthe data at the same timing based on the superposed timing information.10. A data combining apparatus as set forth in claim 9, wherein thesuperposing unit superposes timing information to a blanking period ofthe first data or the second data relating to an image.
 11. A datacombining method combining first data and second data relating to animage to form a screen, the method comprising: a first step ofperforming predetermining processing on the first data; a second step ofgenerating the second data; and a third step of combining the first dataprocessed by the predetermined processing at the first step and thesecond data generated by the second step, and wherein the timinginformation for the processing at the first step or the process forgeneration at the second step is generated so that the combination ofthe first data processed by the predetermined processing at the firststep and the second data generated at the second step is performed atthe same timing in the third step, and at the first step or at thesecond step, the timing for processing or generation is corrected basedon the timing information to perform the predetermined processing or thedata generation.
 12. A data combining method as set forth in claim 11,further comprising: performing processing introducing a delay to thefirst data processed by the predetermined processing at the first stepas a fourth step, and generating the timing information considering theamount of delay of the fourth step.
 13. A data combining method as setforth in claim 11, further comprising generating timing informationincluding a countdown up to the start of a change of screen formation.14. A data combining method as set forth in claim 11, furthercomprising: superposing the generated timing information on the firstdata or the second data, and, at the first step or the second step,correcting the timing of processing or generation to perform thepredetermined processing or the data generation based on the superposedtiming information.
 15. A data combining method as set forth in claim14, further comprising superposing the timing information to a blankingperiod of the first data or the second data relating to an image.
 16. Adata combining method combining first data and second data relating toan image to form a screen comprising: a first step of performingpredetermining processing on the first data; a second step of generatingthe second data; and a third step of combining the first data processedby the predetermined processing at the first step and the second datagenerated by the second step, and wherein at least one of the timinginformation for the processing at the first step and the processing forgeneration at the second step is generated so that the combination ofthe first data processed by the predetermined processing at the firststep and the second data generated at the second step is performed atthe same timing in the third step, and in the third step the data iscorrected and combined at the same timing based on the timinginformation.
 17. A data combining method as set forth in claim 16,further comprising: performing processing introducing a delay to thefirst data processed by the predetermined processing at the first stepas a fourth step and generating the timing information by consideringthe amount of delay in the fourth step.
 18. A data combining method asset forth in claim 16, further comprising generating timing informationincluding a countdown up to the start of a change of screen formation.19. A data combining method as set forth in claim 16, furthercomprising: superposing the generated timing information on the firstdata or the second data and, at the first step or the second step,correcting the timing of processing or generation based on thesuperposed timing information to perform the predetermined processing ordata generation.
 20. A data combining method as set forth in claim 19,further comprising superposing the timing information to a blankingperiod of the first data or the second data relating to an image.